Intravascular catheter for modeling blood vessels

ABSTRACT

A method of generating a 4D model includes capturing imagery of a catheter and a vessel as the catheter is directed through the vessel to a location of interest, wherein the catheter is disposed on a guidewire, constructing a 3D time varying reference curve describing a trajectory of the guidewire, and constructing a time varying 3D model of the artery using the reference curve.

CROSS-REFERENCE TO RELATED APPLICATION

This application is a continuation of U.S. patent application Ser. No.15/498,159, filed Apr. 26, 2017, the complete disclosure of which isexpressly incorporated herein by reference in its entirety for allpurposes.

BACKGROUND

The present disclosure relates generally to an intravascular catheterand to a method for using an intravascular catheter for modeling thegeometry of blood vessels.

Cardiovascular Disease is a leading health problem in the developedworld, with Coronary Artery Disease (CAD) being a particularly importantproblem. CAD typically occurs when part of the smooth, elastic lininginside a coronary artery becomes hardened, stiffened, and swollen withplaque. Plaque typically consists of calcium deposits, fatty deposits,and abnormal inflammatory cells. The formation of a plaque is termedAtherosclerosis. This plaque can cause an obstruction to the supply ofoxygenated blood to the heart muscle and can cause chest pain (angina),and ultimately lead to heart attack and death.

The field of Interventional Cardiology is a branch of cardiology thatdeals specifically with catheter based treatments of structural heartdiseases such as CAD. During a procedure known as Percutaneous CoronaryIntervention (PCI) a catheter is inserted into a major systemic arteryin either the groin or the arm and guided towards the entrance to thecoronary tree. This catheter takes the form of a thin tube, throughwhich a radio-opaque dye may be delivered into the bloodstream, allowingfor visualization of the coronary arteries using a special type of X-rayimaging called Fluoroscopy.

If the vessel narrowing is deemed severe enough, a common treatment isthe insertion of a stent to increase the diameter of the artery. Toplace the stent, another catheter is threaded through the blood vesselsvia a guidewire into the heart where the coronary artery is narrowed byplaque. When the catheter is in place, a balloon tip covered with astent is inflated. The balloon tip compresses the plaque and expands thestent. Once the plaque is compressed and the stent is in place, theballoon is deflated and withdrawn with the catheter. The stent stays inthe artery, holding the artery open. If the stent is not alignedcorrectly with the walls of the artery, termed incomplete stentapposition (ISA), there is an increased risk of long term negativeoutcomes such as myocardial infarction and stent thrombosis.

One of the challenges in this field is accurately assessing the state ofdisease in a patient's coronary arteries. Coronary angiograms arecommonly used by clinicians to manually identify and quantify the degreeof a stenosis, wherever they may appear. Given an angiogram's finiteresolution, the fact that they are 2D projections of a complexthree-dimensional (3D) structure, and the fact that a stenosis canreduce the amount of contrast agent flowing through a stenosis, reducingits visibility, has meant that in many cases other catheter basedimaging techniques are used. These techniques include IntravascularUltrasound (IVUS) or Intravascular Optical Coherence Tomgraphy (OCT),that acquire images of an artery and stenosis from the interior of theartery using an imaging sensor located at the catheter tip. Althoughthese modalities provide much higher resolution images of the interiorof an artery, they do not acquire any information about the 3D positionand orientation of the catheter tip while the images are being acquired(e.g., if an OCT dataset is acquired while retracting the imaging sensorthrough a tortuous curved arterial branch, this could not be ascertainedby examining the images).

BRIEF SUMMARY

According to an embodiment of the present invention, a method ofgenerating a 4-Dimensional model includes capturing imagery of acatheter and a vessel as the catheter is directed through the vessel toa location of interest, wherein the catheter is disposed on a guidewire,constructing a 3-Dimensional time varying reference curve describing atrajectory of the guidewire, and constructing a time varying3-Dimensional model of the artery using the reference curve.

According to an exemplary embodiment of the present invention, a methodof generating a four-dimensional (4D) model during a PercutaneousCoronary Intervention (PCI) procedure comprises capturing imagery of acatheter and a vessel as the catheter is directed through the vessel toa location of interest, wherein the catheter is disposed on a guidewireand comprises a plurality of markers and a monitoring body, and whereinthe imagery is captured from at least two different viewpoints,segmenting the guidewire and the at least one marker from the imageryand back projecting a segmented guidewire and at least one segmentedmarker into a three-dimensional (3D) space to define a time varyingreference curve defining a motion of the monitoring body, recordingaccelerometer and gyroscope data of the catheter using a combination ofsensors as the catheter is pulled away from the location of interest,integrating the accelerometer and gyroscope data in time, predicting alinear and a rotational position of the monitoring body in the imagery,and constructing the 4D model of the monitoring body and the vesselusing the predicted linear and rotational position of the monitoringbody, wherein the 4D model includes a time varying surface describingthe vessel.

According to an embodiment of the present invention, a catheter includesa monitoring body and a pair of markers disposed on the monitoring body,each marker of the pair of markers encircling the monitoring body,wherein the pair of markers are configured to indicate, through theirappearance from a given point of view, an orientation of the monitoringbody.

According to an embodiment of the present invention, a catheter includesan outer sheath, an inner sheath disposed within the outer sheath, aguidewire, a monitoring body disposed within the inner sheath, animaging sensor disposed at a distal end of the monitoring body, and apair of radio-opaque elliptical hoops disposed on the monitoring body,each radio-opaque elliptical hoop of the pair of radio-opaque ellipticalhoops encircling the monitoring body, wherein the pair of radio-opaqueelliptical hoops are configured to indicate, through their appearance ina two-dimensional projection, an orientation of the monitoring body.

As used herein, “facilitating” an action includes performing the action,making the action easier, helping to carry the action out, or causingthe action to be performed. Thus, by way of example and not limitation,instructions executing on one processor might facilitate an actioncarried out by instructions executing on a remote processor, by sendingappropriate data or commands to cause or aid the action to be performed.For the avoidance of doubt, where an actor facilitates an action byother than performing the action, the action is nevertheless performedby some entity or combination of entities.

One or more embodiments of the invention or elements thereof can beimplemented in the form of a computer program product including acomputer readable storage medium with computer usable program code forperforming the method steps indicated. Furthermore, one or moreembodiments of the invention or elements thereof can be implemented inthe form of a system (or apparatus) including a memory, and at least oneprocessor that is coupled to the memory and operative to performexemplary method steps. Yet further, in another aspect, one or moreembodiments of the invention or elements thereof can be implemented inthe form of means for carrying out one or more of the method stepsdescribed herein; the means can include (i) hardware module(s), (ii)software module(s) stored in a computer readable storage medium (ormultiple such media) and implemented on a hardware processor, or (iii) acombination of (i) and (ii); any of (i)-(iii) implement the specifictechniques set forth herein.

Techniques of the present invention can provide substantial beneficialtechnical effects. For example, one or more embodiments may provide oneor more of the following advantages:

-   -   Provide accurate visualization of the morphology of a patient's        coronary arteries, combining both the 3D time varying nature of        the arteries, with a high resolution view of the arterial wall        and its composition.    -   Allow for high resolution patient specific computer simulation        of hemodynamics through a stenosis or a deployed stent, to        assess ISA and disruption to normal blood flow and provide a        means for predicting restenosis.    -   Enable personalized stent design and deployment, reducing the        risk of ISA and thereby improving outcomes for patients.

These and other features and advantages of the present invention willbecome apparent from the following detailed description of illustrativeembodiments thereof, which is to be read in connection with theaccompanying drawings.

BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWINGS

Preferred embodiments of the present invention will be described belowin more detail, with reference to the accompanying drawings:

FIG. 1 is a diagram of a catheter according to an embodiment of thepresent invention;

FIGS. 2A-C are perspective views of a geometric array of radio-opaquemarkers according to an embodiment of the present invention;

FIG. 3 is a block diagram of a catheter according to an embodiment ofthe present invention;

FIG. 4 is a view of two sets of angiograms acquired from differentviewpoints according to an embodiment of the present invention;

FIG. 5 is an illustration of a guidewire and markers segmented fordefining a 4D trajectory according to an embodiment of the presentinvention;

FIG. 6 is an illustration of a pullback procedure according to anembodiment of the present invention;

FIG. 7 is a block diagram of a method of building a 4D model accordingto an embodiment of the present invention;

FIG. 8 is a block diagram of a method of building a 4D model accordingto an embodiment of the present invention; and

FIG. 9 is a block diagram depicting an exemplary computer systemembodying a method for modeling a blood vessel according to an exemplaryembodiment of the present invention.

DETAILED DESCRIPTION

According to an embodiment of the present invention, a catheter isconfigured to record an orientation of its tip as it moves through acoronary artery. According to an embodiment of the present invention, adirection vector is defined for the tip of the catheter and optionally,a measure of twist of the catheter. According to an embodiment of thepresent invention, this information is combined with imaging modalitiessuch as Intravascular Ultrasound (IVUS) and intravascular OpticalCoherence Tomography (OCT), enabling a reconstruction of the arteriallumen. According to an embodiment of the present invention, a techniquefor determining the 3D position and orientation of an imaging catheterduring a pullback provides technical advantages including enabling thedetails of the arterial wall and its composition to be positionedcorrectly.

FIG. 1 is a diagram of a catheter 100 according to an embodiment of thepresent invention, wherein the catheter 100 comprises an outer sheath101, an inner sheath 102 a guidewire 103, a monitoring body 104, asensor 105 (e.g., an imaging sensor) attached to the monitoring body104, a first marker 106 (e.g., a lens marker), a second or proximalmarker 107, a distal tip marker 108 and a distal tip 109 of theguidewire 103. The catheter 100 further comprises a geometric array 110,described herein.

The catheter 100 is configured for acquiring IVUS or OCT images, whereinthe outer sheath 101 remains in place during a pullback of the innersheath 102, guidewire 103 and monitoring body 104, including the sensor105, through the coronary artery (not shown). Furthermore, the markers106-108 and 110 can be observed in an angiogram.

According to an embodiment of the present invention, the geometric array110 includes radio-opaque markers 201-202. The radio-opaque markers201-202 can be retro-fitted onto existing catheters to indicate twist ofthe catheter. These radio-opaque markers 201-202 are arranged in afashion that enables determination of the orientation of the monitoringbody 104. The radio-opaque markers 201-202 include 2 elliptical hoops(201 and 202, respectively) affixed to the exterior of the monitoringbody, such that they remain stationary with respect to the imagingmodality (OCT, or IVUS) The radio-opaque markers 201-202 are disposedorthogonal to one another, and at a fixed distance from the markers106-108. Since the entirety of each loop is radio-opaque, a full ellipseis visible in the angiogram (or other imaging method) from all angles,unless that angle is parallel to the plane of the ellipse, in which casea single line is seen in the angiogram. Each orientation of the catheterfrom a particular viewpoint is associated via a lookup table with aunique 2D projection. Therefore, given a particular marker 2D projectionseen on the angiogram, the pattern of the markers in relation to theangle of view of the angiogram relates to a single angle of twist of thecatheter tip.

In addition to the geometric array 110, according to an embodiment ofthe present invention, the catheter 100 includes an Inertial MeasurementUnit (IMU) configured to record orientation data relative to a datumpoint. The IMU provides linear and rotational information about thecatheter, enabling the determination of a trajectory, which can beperiodically confirmed in the angiogram via the orientation of themarkers. FIG. 3 shows the monitoring body 104 of the catheter 100including the geometric array 110, which is positioned on the devicealongside the IMU 301. The IMU 301 includes one or more of a gyroscope302 (such as an optical gyroscope), an accelerometer 303, and amagnetometer 304, positioned on the monitoring body, at the cathetertip. According to an embodiment of the present invention, 3 orthogonalaccelerometers and 3 orthogonal gyroscopes are included in the IMU inorder to measure inertial acceleration and rotation position,respectively, in 3 dimensions. The sensors 302-304 can be placed in anycombination possible alongside the sensor 105 and first marker 106. Itshould be appreciated that other configurations of sensors arecontemplated, and that these configurations would be understood a personof ordinary skill in the art in view of the present disclosure.

According to an embodiment of the present invention, a method 800 (seeFIG. 8) of generating a four-dimensional (4D) model during a PCIprocedure includes threading a catheter (incorporating an imagingsensor, IMU, and specialized markers) onto a guidewire and directing thecatheter through a coronary artery to a location of interest (e.g.,stenosis or otherwise) 801. According to an embodiment of the presentinvention, the method includes capturing imagery (e.g., angiograms) 802from two different viewpoints simultaneously using bi-plane angiographyover a single cardiac cycle. According to an embodiment of the presentinvention, the method includes capturing imagery (e.g., angiograms) 802from two different viewpoints, one at a time (using single planeangiography to view from two viewpoints over different cycles)(401-402), for at least one cardiac cycle as illustrated in FIGS. 4-5.In the case of a scanner without bi-lane imaging capabilities, a firstangiogram is acquired using a scanner, a gantry supporting the scanneris rotated changing the view point, and a second angiogram acquiredusing the scanner. According to an embodiment of the present invention,the guidewire 403 and the markers, e.g., 404, (including the markers106-108 and geometric array 110) are visible in the imagery.

According to an embodiment of the present invention, the method furthercomprises processing the imagery 803. More particularly, using these twoangiograms 401-402, the guidewire 403 and markers 404 are segmented ineach frame and projected into 3D space to define a time varyingreference curve defining the motion of the monitoring body, which isaffixed to the catheter, as a sequence of coordinates in space and time(i.e., a 4D trajectory) (see FIG. 5). In addition, the rotationalposition of the imaging sensor, relative to the angiogram viewpoint, canbe determined through observation of a marker pattern of the geometricarray 110 in each angiogram frame, wherein both the initial linear androtational positions of the imaging sensor can be defined and specifiedas an initial condition.

According to an embodiment of the present invention, the segmentationcan include extracting a coronary artery tree from each of a sequence ofangiogram images, creating a mean artery tree from the extractedcoronary artery trees, and projecting the mean artery tree back ontoeach of the sequence of angiogram images to recover a complete coronaryartery tree for each of the sequence of angiogram images. Here, thedisclosure of U.S. patent application Ser. No. 14/243,106, entitledDetecting Coronary Stenosis Through Spatio-Temporal Tracking, and filedon Apr. 2, 2014, is incorporated by reference in its entirety. It shouldbe understood that other methods of segmenting the guidewire and markerscan be used, for example, rule based classification of pixels of theframes and deep learning/neutral network classifiers can be used in oneor more embodiments of the present invention.

FIG. 5 is an illustration of the definition of the time varying 3Dreference curve 501. Using two angiogram runs (502, 503) acquired fromdifferent perspectives (504, 505) the proximal marker (e.g., 506 a-b)and distal tip markers (e.g., 507 a-b), the lens marker (e.g., 508 a-b),and guidewire (e.g., 510 a-b) can be segmented and triangulated into theglobal world coordinate system (i.e., the one in an x,y,z coordinatesystem). The time varying 3D reference curve 501 includes the location(over the cardiac system indicated by arrow 511) of the proximal marker506 c, the distal top marker 507 c, the lens marker 508 c and theguidewire 510 c. Curve 512 illustrates an ECG trace (available for eachrun), which is used to map angiogram frames to corresponding positionsin the cardiac cycle (where it is assumed that the coronary geometrywould be in the same physical position in the world coordinate system).The different perspectives correspond to imaging devices (e.g., an X-Raysource) used to acquire the runs.

According to an embodiment of the present invention, the processing 803can be performed using existing techniques. In one example, theguidewire 403, proximal marker 404, lens marker 405 and distal marker406 are extracted from each of a sequence of angiogram images ofangiograms 401-402 by estimating a position of the guidewire 403 andmarkers 404-406 from each of the sequence of angiogram images. Meanpositions of the guidewire 403 and markers 404-406 are calculated andre-aligned back to each of the sequences of angiogram images. Thisresults in a set of positions widths along a common set of arterysegments that can be compared over time.

The method further includes recording accelerometer and gyroscope datausing some combination of sensors 302-304 as the catheter is wound back804. This data includes a measure of inertial acceleration androtational position in 3 dimensions. By integrating the accelerometerand gyroscope data in time based on the initial condition 804, apredicted linear and rotational position of the imaging sensor isestimated 805, to where the acquired frame can be located (see FIG. 6).More particularly, according to an embodiment of the present invention,the pullback has constant rate while the sensor 105 acquires images at aspecified frame rate.

FIG. 6 is an illustration of the pullback procedure and the mapping ofarterial cross sections obtained from the intravascular images, onto thereference curve. More particularly FIG. 6 illustrates a heart rhythmrecording 601 with indications, e.g., 602, corresponding to each frameof the dataset, e.g., 603. The heart rhythm recording 601 illustratesthe ECG trace acquired with the angiogram during the pullback and theindications, e.g., 602, depict the positions in the multiple cardiaccycles over which images of the pullback procedure are acquired.Similarly, indication 602 corresponds to marker 604 in the mapping. Inthe mapping, the rotation position of the image sensor is illustrated bythe ovals, e.g., 605, which correspond to the markers contained therein.The ovals, e.g., 605, and markers, e.g., 604, illustrate the arterialwall cross section and centroid respectively, segmented from each image,e.g., 603, during the pullback, which are subsequently mapped topositions along the reference curve, defining a time varying geometry.Time-varying anatomy (e.g., changing shape of the artery during thecardiac cycle) is indicated by arrow 606. The curves (e.g., 607) andmarkers (e.g., 608) illustrate the guidewire and the time varyingposition of the proximal and distal markers (see for example, FIG. 5,markers 506, 507), respectively.

According to an embodiment of the present invention, the method includescorrecting for the drift error 806 accumulated in the accelerometerrecording, using magnetometer data. According to an embodiment of thepresent invention, a filtering approach using the predicted and observedpositions of the monitoring body is applied. By making use of the factthat the imaging sensor is constrained to move along the guidewire, theinitial position and the known pullback rate are used to compute alocation along the reference curve at the moment in time that the framewas acquired. Since the moment in time will not likely coincide with apoint in time at which the reference curve is defined, the position isinterpolated between neighboring points in space and time on thereference curve. At this stage a filtered estimate of the linear andangular position of the imaging sensor is determined and the data can belocked on to that point on the reference curve and oriented such thatthe image normal vector is parallel to the reference curve tangent atthat point. While the image frames acquired by OCT or IVUS show thevessel cross-section of each frame location at a single point in thecardiac cycle, it is assumed that the cross-section at that vessellocation does not change throughout the cycle, and remains stationary inrelation to the associated point on the reference curve.

At this point the data acquisition is complete and the model isconstructed 807. According to an exemplary embodiment of the presentinvention, the model is built by applying known segmentation techniquesto extract contours from each frame defining the arterial wall andstent, applying or generating of point cloud data that may be used todefine a time varying surface describing the arterial wall. The outcomeof the method depicted in FIG. 8 is a model defining the 4 dimensionalstructure of the blood vessel in question, including the cross-sectionalong the curve, and movement of the vessel throughout the cardiaccycle.

Consider the following exemplary implementation illustrated in themethod 700 of FIG. 7; according to one or more embodiments of thepresent invention, with a patient lying supine in X-Ray imaging systemand a guidewire navigated through a branch/stenosis of interest, such asin as Percutaneous Coronary Intervention procedure, a 3D time varyingreference curve describing the trajectory of the guidewire disposed inan artery is constructed at 701, and a high-resolution time varying 3Dcomputer model of the artery is constructed at 702.

According to an exemplary embodiment of the present invention, theconstruction of the 3D time varying reference curve describing thetrajectory of the guidewire 701 uses image segmentation to extractpixels describing the guidewire from each angiogram, and performs amapping to a world coordinate system to output the time varying 3Dreference curve. According to an exemplary embodiment of the presentinvention, the time varying 3D reference curve is the trajectory of theguidewire. The trajectory of the guidewire is used to constraincalculations about the movement of the imaging sensor (and/or othercomponents of the catheter) during pullback.

According to an exemplary embodiment of the present invention, as the 3Dtime varying reference curve describing the trajectory of the guidewireis built 701, an initial position and orientation of the lens marker isdefined using image segmentation to extract pixels describing theelliptical markers, which are then mapped to the world coordinatesystem. The method uses at least two different angiograms taken fromdifferent viewpoints, to reconstruct the 3D world position of theelliptical markers. According to one or more embodiments of the presentinvention, it is assumed that the catheter and guidewire move in aperiodic motion over time, where the motion of the guidewire from thefirst to the second scan corresponds with the motion of the heart.According to an exemplary embodiment of the present invention, whenperforming the pullback in the context of a non-bi-plane scanner, onlyone angiogram is acquired and used to track the lens marker. It shouldbe understood that a bi-plane scanner can be used during pullback, andmay provide additional imaging information. Given the 3D time varyingreference curve, an image segmentation is performed on the angiogram(s)acquired during the pullback, and the position of the imaging sensor ismapped the onto the 3D time varying reference curve 702. According to anembodiment of the present invention, a lookup table is used fordetermining the orientation of the sensor based on the segmentation ofthe elliptical markers, where the lookup table includes data about knownimagery of the elliptical markers corresponding to differentorientations of the sensor.

According to an exemplary embodiment of the present invention, and withreference to FIG. 7, the construction of the 3D time varying referencecurve describing the trajectory of the guidewire 701 comprises acquiringat least two angiogram runs from different viewing angles 711 (e.g.,greater than 25 degrees apart, either with a monoplane or biplane X-Rayimaging system) with the guidewire, and the proximal marker, the distaltip marker, and the lens marker visible over a complete cardiac cycle(e.g., with no dye injected), mapping each frame to positions within thecardiac cycle (e.g., using well-known signal processing techniques tolocate the QRS complex seen on a typical electrocardiogram) using theECG data embedded in the runs as metadata 712 and selecting a subset offrames from each run that encompass at least one complete cardiac cycle713, applying image/video processing analytics to the selected framesfrom each run to segment the markers and guidewire (e.g., using a methodsuch as Convolutional Neural Networks (CNN) trained to perform objectdetection on the markers and semantic pixel labeling on the guidewire)714, using the well-known imaging parameters embedded in the runs asmetadata (e.g. primary (left and right anterior oblique views, LAO andRAO, respectively) and secondary (caudal and cranial view, CRAN andCAUD, respectively) viewing angles, distance from source to intensifier(SID) and source to isocenter (SOD), pixel spacing) compute theintersection of the marker points and points along the guidewire in theisocenter world coordinate system (e.g., using triangulation) over time715, in determining the trajectory of the lens marker in the worldcoordinate system as a function of time 716.

According to an exemplary embodiment of the present invention, theconstruction of the high-resolution time varying 3D computer model ofthe artery 702 comprises performing an intravascular imaging pullback ata user specified and controlled rate during which time IMU linear androtational accelerations are recorded 721, acquiring, during thepullback, at least one angiogram run from one of the viewing angles usedto build the reference curve in 701, such that the lens marker isvisible 722 (e.g., with no dye injected), using the ECG data embedded inthe run as metadata to map each frame to a position within the cardiaccycle 723, applying image/video processing analytics to segment the lensmarker in each frame (e.g., using a CNN trained to perform semanticpixel labeling on the lens marker such that the appearance of theelliptical markers can be observed) 724, using a known pullback rate(and given the fact that the imaging sensor is constrained to move alongthe guidewire), compute distance of lens marker along reference curverelative to starting position (e.g., a start of the pullback, a start ofthe recording, etc.), at each step during pullback 725, and usingobserved appearance of elliptical markers, compute orientation ofcatheter in imaging frame of reference and transform this to the worldcoordinate system 726. At this point there are a set of observedpositions and orientations from each angiogram frame acquired during thepullback. The method further comprises using the world coordinateposition of the lens marker at the time the pullback began to integratethe linear and rotational accelerations forward in time, to integratethis modeled trajectory with the observed trajectory obtained from theangiogram run and reference curve (e.g., using a Kalman filter) 727. Themethod further comprises applying image processing analytics to eachframe of the intravascular pullback (e.g. using a CNN, or level setmethod to extract the arterial wall, stent, etc) to obtain crosssectional curves 728, mapping each extracted cross section to thecorresponding point along the reference curve and connect them (e.g., bydefining triangles between sets of points on adjacent cross sections)729 to obtain the time varying 3D surface model 730.

According to an exemplary embodiment of the present invention, the lensmarker 106 is one of the elliptical markers 201-202. In a case where thelens marker 106 is a ring shaped marker as shown in FIG. 1, separateanalytics can be used to track the ring shaped lens marker 106 and theelliptical markers 201-202. In at least one exemplary embodiment of thepresent invention, the ring shaped lens marker 106 is omitted. Becausethe imaging sensor, lens marker 106 (if present) and elliptical markers201-202 are rigidly connected, tracking the elliptical markers 201-202can yield sufficient tracking information, without separately trackingthe lens marker 106.

Recapitulation:

According to an embodiment of the present invention, a method ofgenerating a 4-Dimensional model includes capturing imagery of acatheter and a vessel as the catheter is directed through the vessel toa location of interest, wherein the catheter is disposed on a guidewire,constructing a 3-Dimensional time varying reference curve describing atrajectory of the guidewire, and constructing a time varying3-Dimensional model of the artery using the reference curve.

Medical images typically adhere to the Digital Imaging andCommunications in Medicine (DICOM) format once they are transferred froman imaging system that acquires them to a database system (e.g., PictureArchiving and Communication System (PACS)). At this point, metadataregarding medical images (e.g., angiogram RAO-LAO, SID, pixel spacing,etc., or OCT pixel spacing, etc.) can be embedded in appropriate DICOMheader fields. According to an embodiment of the present invention,rotational information about a catheter is encoded in medical images.According to at least one embodiment of the present invention, the IMUacceleration and IVUS/OCT imaging data are intercepted (e.g., from theimaging system) and a model is generated before outputting data (e.g.,the IMU acceleration and IVUS/OCT imaging data) to the PACS in DICOMformat. According to an embodiment of the present invention, therotational information about a catheter is used in building a 3D modelof an artery. According to an embodiment of the present invention, themodel so built improves the accuracy of computer simulations ofhemodynamics (e.g. virtual FFR), leads to improved visualizations ofstent placement (improving outcomes for patients), allows for wall shearstress to be determined on a patient specific basis (stenosis form inareas of low shear stress) as a possible predictor of restenosis, andenables personalized stent design and deployment.

According to an embodiment of the present invention, an accelerometerand a gyroscope are disposed at the tip of the catheter, near a lasermarking system 105. The rotational position of the catheter is recordedand matched to the position of the laser. Integration of thisinformation allows for a complete trajectory of the laser tip to bedetermined.

According to an embodiment of the present invention, a catheter havingan accelerometer and a gyroscope enables improved visualization ofcomplex stenotic geometry, side branches, stents, and allow for thegeneration of high resolution 3D computer models of vascular geometriesthat could be used for computer simulations of blood flow, wall shearstress, generation of personalized stent designs, of stent placement,and once deployed, assessment of their impact on normal haemodynamics.

Embodiments of the present invention utilize high-resolution imagingmodalities such as IVUS and intravascular OCT, and is able to determinespatial position and orientation as the catheter steered through thecardiovascular system, building a 4D (i.e., 3D+time varying)representation of the coronary geometry.

Embodiments of the present invention utilize IVUS imaging modality toassess the composition of plaque, thereby informing the treatment plan,and providing information regarding vessel wall and plaque thickness tothe 4D model.

According to an embodiment of the present invention, a method ofgenerating a four-dimensional (4D) model during a Percutaneous CoronaryIntervention (PCI) procedure includes capturing imagery of a catheterand a vessel 802 as the catheter is directed through the vessel to alocation of interest, wherein the catheter is disposed on a guidewireand comprises a plurality of markers and a monitoring body, and whereinthe imagery is captured from at least two different viewpoints,segmenting the guidewire and the at least one marker from the imageryand back projecting a segmented guidewire and at least one segmentedmarker into a three-dimensional (3D) space to define a time varyingreference curve defining a motion of the monitoring body 803, recordingaccelerometer and gyroscope data of the catheter using a combination ofsensors as the catheter is pulled away from the location of interest803, integrating the accelerometer and gyroscope data in time 804,predicting a linear and a rotational position of the monitoring body inthe imagery 805, and constructing the 4D model of the monitoring bodyand the vessel using the predicted linear and rotational position of themonitoring body, wherein the 4D model includes a time varying surfacedescribing the vessel 807.

According to an exemplary embodiment of the present invention, thetrajectory of the guidewire is used to determine the position of thelens marker and imaging sensor. Embodiments of the present inventionovercome a problem in the art where an imaging system (e.g., X-rayimaging system), a catheter imaging system and an IMU are notsynchronized to provide measurements at the same points in time.Embodiments of the present invention facilitate the positioning ofimages along a reference curve at correct times, at the times that theimages were acquired. Exemplary embodiments of the present inventionintegrate acceleration data forward in time to predict a trajectory ofthe lens marker and imaging sensor, with periodic corrections to thesepredictions given an angiogram frame where the position is actuallyobserved.

The methodologies of embodiments of the disclosure may be particularlywell-suited for use in an electronic device or alternative system.Accordingly, embodiments of the present invention may take the form ofan entirely hardware embodiment or an embodiment combining software andhardware aspects that may all generally be referred to herein as a“processor,” “circuit,” “module” or “system.”

Furthermore, it should be noted that any of the methods described hereincan include an additional step of providing a computer system forgenerating a 4D model during a PCI procedure. Further, a computerprogram product can include a tangible computer-readable recordablestorage medium with code adapted to be executed to carry out one or moremethod steps described herein, including the provision of the systemwith the distinct software modules.

Referring to FIG. 9; FIG. 9 is a block diagram depicting an exemplarycomputer system embodying the computer system for generating a 4D modelduring a PCI procedure according to an embodiment of the presentinvention. The computer system shown in FIG. 9 includes a processor 901,memory 902, display 903, input device 904 (e.g., keyboard), a networkinterface (I/F) 905, a media I/F 906, and media 907, such as a signalsource, e.g., camera, Hard Drive (HD), external memory device, etc.

In different applications, some of the components shown in FIG. 9 can beomitted. The whole system shown in FIG. 9 is controlled by computerreadable instructions, which are generally stored in the media 907. Thesoftware can be downloaded from a network (not shown in the figures),stored in the media 907. Alternatively, software downloaded from anetwork can be loaded into the memory 902 and executed by the processor901 so as to complete the function determined by the software.

The processor 901 may be configured to perform one or more methodologiesdescribed in the present disclosure, illustrative embodiments of whichare shown in the above figures and described herein. Embodiments of thepresent invention can be implemented as a routine that is stored inmemory 902 and executed by the processor 901 to process the signal fromthe media 907. As such, the computer system is a general-purposecomputer system that becomes a specific purpose computer system whenexecuting routines of the present disclosure.

Although the computer system described in FIG. 9 can support methodsaccording to the present disclosure, this system is only one example ofa computer system. Those skilled of the art should understand that othercomputer system designs can be used to implement embodiments of thepresent invention.

The present invention may be a system, a method, and/or a computerprogram product at any possible technical detail level of integration.The computer program product may include a computer readable storagemedium (or media) having computer readable program instructions thereonfor causing a processor to carry out aspects of the present invention.

The computer readable storage medium can be a tangible device that canretain and store instructions for use by an instruction executiondevice. The computer readable storage medium may be, for example, but isnot limited to, an electronic storage device, a magnetic storage device,an optical storage device, an electromagnetic storage device, asemiconductor storage device, or any suitable combination of theforegoing. A non-exhaustive list of more specific examples of thecomputer readable storage medium includes the following: a portablecomputer diskette, a hard disk, a random access memory (RAM), aread-only memory (ROM), an erasable programmable read-only memory (EPROMor Flash memory), a static random access memory (SRAM), a portablecompact disc read-only memory (CD-ROM), a digital versatile disk (DVD),a memory stick, a floppy disk, a mechanically encoded device such aspunch-cards or raised structures in a groove having instructionsrecorded thereon, and any suitable combination of the foregoing. Acomputer readable storage medium, as used herein, is not to be construedas being transitory signals per se, such as radio waves or other freelypropagating electromagnetic waves, electromagnetic waves propagatingthrough a waveguide or other transmission media (e.g., light pulsespassing through a fiber-optic cable), or electrical signals transmittedthrough a wire.

Computer readable program instructions described herein can bedownloaded to respective computing/processing devices from a computerreadable storage medium or to an external computer or external storagedevice via a network, for example, the Internet, a local area network, awide area network and/or a wireless network. The network may comprisecopper transmission cables, optical transmission fibers, wirelesstransmission, routers, firewalls, switches, gateway computers and/oredge servers. A network adapter card or network interface in eachcomputing/processing device receives computer readable programinstructions from the network and forwards the computer readable programinstructions for storage in a computer readable storage medium withinthe respective computing/processing device.

Computer readable program instructions for carrying out operations ofthe present invention may be assembler instructions,instruction-set-architecture (ISA) instructions, machine instructions,machine dependent instructions, microcode, firmware instructions,state-setting data, configuration data for integrated circuitry, oreither source code or object code written in any combination of one ormore programming languages, including an object oriented programminglanguage such as Smalltalk, C++, or the like, and procedural programminglanguages, such as the “C” programming language or similar programminglanguages. The computer readable program instructions may executeentirely on the user's computer, partly on the user's computer, as astand-alone software package, partly on the user's computer and partlyon a remote computer or entirely on the remote computer or server. Inthe latter scenario, the remote computer may be connected to the user'scomputer through any type of network, including a local area network(LAN) or a wide area network (WAN), or the connection may be made to anexternal computer (for example, through the Internet using an InternetService Provider). In some embodiments, electronic circuitry including,for example, programmable logic circuitry, field-programmable gatearrays (FPGA), or programmable logic arrays (PLA) may execute thecomputer readable program instructions by utilizing state information ofthe computer readable program instructions to personalize the electroniccircuitry, in order to perform aspects of the present invention.

Aspects of the present invention are described herein with reference toflowchart illustrations and/or block diagrams of methods, apparatus(systems), and computer program products according to embodiments of theinvention. It will be understood that each block of the flowchartillustrations and/or block diagrams, and combinations of blocks in theflowchart illustrations and/or block diagrams, can be implemented bycomputer readable program instructions.

These computer readable program instructions may be provided to aprocessor of a general purpose computer, special purpose computer, orother programmable data processing apparatus to produce a machine, suchthat the instructions, which execute via the processor of the computeror other programmable data processing apparatus, create means forimplementing the functions/acts specified in the flowchart and/or blockdiagram block or blocks. These computer readable program instructionsmay also be stored in a computer readable storage medium that can directa computer, a programmable data processing apparatus, and/or otherdevices to function in a particular manner, such that the computerreadable storage medium having instructions stored therein comprises anarticle of manufacture including instructions which implement aspects ofthe function/act specified in the flowchart and/or block diagram blockor blocks.

The computer readable program instructions may also be loaded onto acomputer, other programmable data processing apparatus, or other deviceto cause a series of operational steps to be performed on the computer,other programmable apparatus or other device to produce a computerimplemented process, such that the instructions which execute on thecomputer, other programmable apparatus, or other device implement thefunctions/acts specified in the flowchart and/or block diagram block orblocks.

The flowchart and block diagrams in the Figures illustrate thearchitecture, functionality, and operation of possible implementationsof systems, methods, and computer program products according to variousembodiments of the present invention. In this regard, each block in theflowchart or block diagrams may represent a module, segment, or portionof instructions, which comprises one or more executable instructions forimplementing the specified logical function(s). In some alternativeimplementations, the functions noted in the blocks may occur out of theorder noted in the Figures. For example, two blocks shown in successionmay, in fact, be executed substantially concurrently, or the blocks maysometimes be executed in the reverse order, depending upon thefunctionality involved. It will also be noted that each block of theblock diagrams and/or flowchart illustration, and combinations of blocksin the block diagrams and/or flowchart illustration, can be implementedby special purpose hardware-based systems that perform the specifiedfunctions or acts or carry out combinations of special purpose hardwareand computer instructions.

The terminology used herein is for the purpose of describing particularembodiments only and is not intended to be limiting of the invention. Asused herein, the singular forms “a”, “an” and “the” are intended toinclude the plural forms as well, unless the context clearly indicatesotherwise. It will be further understood that the terms “comprises”and/or “comprising,” when used in this specification, specify thepresence of stated features, integers, steps, operations, elements,and/or components, but do not preclude the presence or addition of oneor more other features, integers, steps, operations, elements,components, and/or groups thereof.

The corresponding structures, materials, acts, and equivalents of allmeans or step plus function elements in the claims below are intended toinclude any structure, material, or act for performing the function incombination with other claimed elements as specifically claimed. Thedescription of the present invention has been presented for purposes ofillustration and description, but is not intended to be exhaustive orlimited to the invention in the form disclosed. Many modifications andvariations will be apparent to those of ordinary skill in the artwithout departing from the scope and spirit of the invention. Theembodiment was chosen and described in order to best explain theprinciples of the invention and the practical application, and to enableothers of ordinary skill in the art to understand the invention forvarious embodiments with various modifications as are suited to theparticular use contemplated.

What is claimed is:
 1. A method of generating a 4-Dimensional modelcomprising: capturing imagery of a catheter and a vessel as the catheteris directed through the vessel to a location of interest, wherein thecatheter is disposed on a guidewire; constructing a 3-Dimensional timevarying reference curve describing a trajectory of the guidewire; andconstructing a time varying 3-Dimensional model of the artery using thereference curve.
 2. The method of claim 1, wherein constructing the3-Dimensional time varying reference curve describing a trajectory ofthe guidewire further comprises: acquiring at least two angiogram runsfrom different viewing angles with the guidewire, and a proximal marker,a distal tip marker, and a lens marker visible over a complete cardiaccycle; mapping each frame of the at least two angiogram runs topositions within the cardiac cycle; selecting a subset of the frames ofthe at least two angiogram runs that encompass at least the cardiaccycle; segmenting the proximal marker, the distal tip marker, the lensmarker and the guidewire from the subset of the frame; and computing anintersection between the at least two angiogram runs for each of theproximal marker, the distal tip marker, the lens marker and theguidewire in a world coordinate system over time to determine areference curve as a trajectory of the lens marker in the worldcoordinate system as a function of time.
 3. The method of claim 1,wherein constructing the time varying 3-Dimensional model of the arteryusing the reference curve further comprises: performing an intravascularimaging pullback at a known rate during while recording linear androtational accelerations of the catheter; acquiring, during theintravascular imaging pullback, at least one angiogram run from aviewing angle used in determining the trajectory of the guidewire;mapping each frame of the at least one angiogram run to a positionwithin a cardiac cycle; segmenting a lens marker of the catheter in eachof the frames; computing a distance of the lens marker along thetrajectory of the guidewire relative to starting position at a pluralityof steps of the intravascular imaging pullback using the known rate ofthe intravascular imaging pullback; transforming an orientation of thelens marker of the catheter in each of the frames to a world coordinatesystem; integrate the linear and rotational accelerations of thecatheter forward in time with the trajectory of the guidewire using theworld coordinate position of the lens marker; obtaining a plurality ofcross sectional curves of each of the frames; mapping each of the crosssectional curves to a corresponding point along the trajectory of theguidewire; and obtaining a time varying 3D surface model by connectingthe cross sectional curves.
 4. A method of generating a four-dimensional(4D) model during a Percutaneous Coronary Intervention (PCI) procedurecomprising: capturing imagery of a catheter and a vessel as the catheteris directed through the vessel to a location of interest, wherein thecatheter is disposed on a guidewire and comprises a plurality of markersand a monitoring body, and wherein the imagery is captured from at leasttwo different viewpoints; segmenting the guidewire and at least onemarker from the imagery and back projecting a segmented guidewire and atleast one segmented marker into a three-dimensional (3D) space to definea time varying reference curve defining a motion of the monitoring body;recording accelerometer and gyroscope data of the catheter using acombination of sensors as the catheter is pulled away from the locationof interest; integrating the accelerometer and gyroscope data in time;predicting a linear and a rotational position of the monitoring body inthe imagery; and constructing the 4D model of the monitoring body andthe vessel using the predicted linear and rotational position of themonitoring body, wherein the 4D model includes a time varying surfacedescribing the vessel.
 5. The method of claim 4, wherein capturingimagery comprises capturing a plurality of angiograms.
 6. The method ofclaim 4, wherein capturing imagery comprises capturing the differentviewpoints simultaneously over a single cardiac cycle.
 7. The method ofclaim 4, wherein capturing imagery comprises capturing the differentviewpoints one at a time over consecutive cardiac cycles.
 8. The methodof claim 4, wherein the time varying reference curve defines the motionof the monitoring body as a sequence of coordinates in space and time.9. The method of claim 4, wherein the accelerometer and gyroscope datacomprises a measure of inertial acceleration and rotational position inthree dimensions.
 10. The method of claim 4, further comprisingcorrecting for a drift error accumulated in the accelerometer data usingmagnetometer data.